Generalization by structural properties from sparse nested symbolic data
نویسنده
چکیده
A set of simulations demonstrate that recurrent networks can exhibit generalization by abstraction from extremely sparse but structurally homogenous symbolic data. By cascading two recurrent networks – feeding the second network with discretized hidden states of the first – it is also possible to generalize according to complex structure. By automatic discretization the cascaded architecture assists in scaling up sequential learning tasks and offers explanations to the apparent systematicity and generativity of language use.
منابع مشابه
Generalization by symbolic abstraction in cascaded recurrent networks
Generalization performance in recurrent neural networks is enhanced by cascading several networks. By discretizing abstractions induced in one network, other networks can operate on a coarse symbolic level with increased performance on sparse and structural prediction tasks. The level of systematicity exhibited by the cascade of recurrent networks is assessed on the basis of three language doma...
متن کاملSolving Linear Systems with Sparse Matrices on Hypercubes
We investigate parallel Gauss elimination for sparse matrices, especially those arising from the discretization of PDEs. We propose an approach which combines minimum degree ordering, nested dissection, domain decomposition and multifront techniques. Neither symbolic factorization nor explicit representation of elimination trees are needed. An effective and economic dynamic data structure is pr...
متن کاملGlobal Astrometric Solutions with Sparse Matrix Techniques
Modern astrometric techniques lead to large, linear systems solved by the precepts of least-squares. These systems are usually sparse, and one should take advantage of the sparsity to facilitate their solution. As long as the matrix A of the equations of condition possesses the weak Hall property, characteristic of linear systems derived from astrometric reductions, it is possible to nd a spars...
متن کاملPerformance of a Fully Parallel Sparse Solver
The performance of a fully parallel direct solver for large sparse symmetric positive deenite systems of linear equations is demonstrated. The solver is designed for distributed-memory, message-passing parallel computer systems. All phases of the computation, including symbolic processing as well as numeric factorization and triangular solution, are performed in parallel. A parallel Cartesian n...
متن کاملSymbolic Visibly Pushdown Automata
Nested words model data with both linear and hierarchical structure such as XML documents and program traces. A nested word is a sequence of positions together with a matching relation that connects open tags (calls) with the corresponding close tags (returns). Visibly Pushdown Automata are a restricted class of pushdown automata that process nested words, and have many appealing theoretical pr...
متن کامل